Conventional Maximum Likelihood Estimation Research Articles

A new estimation approach based on phi‐divergence measures for one‐shot device accelerated life testing

AbstractOne‐shot device testing data are used only once and they get destroyed when tested. As these products usually have large mean times to failure under normal operating conditions, accelerated life tests are commonly used to infer their lifetime distribution. While much work has been done to determine the maximum likelihood estimates (MLEs) of model parameters for one‐shot device accelerated life testing, the efficiency of these methods may not be guaranteed for small to moderate sample sizes. In this paper, we develop new estimators and confidence intervals based on phi‐divergences, we show that they outperform the conventional MLE under different lifetime distributions and present an example to illustrate all the inferential methods developed here.

Classical and Bayesian Inference of Unit Gompertz Distribution Based on Progressively Type II Censored Data

In this article, two frequentist approaches and a Bayesian approach employing progressive Type II censored data are used to estimate parameters of a unit Gompertz distribution. In frequentist approach, besides conventional maximum likelihood estimation, maximum product of spacing method is proposed for parameter estimation as an alternative approach to common maximum likelihood method. Both Newton-Raphson and stochastic expectation minimization algorithms are used for computing the MLEs, while Bayes estimates are obtained using both the product of spacing function and the likelihood function. Additionally, the highest posterior density (HPD) credible intervals are compared with the approximate confidence intervals (CIs) for the parameters of the model that were derived using both traditional approaches. Moreover, percentile bootstrap technique is utilized to compute confidence intervals. Numerical comparisons are presented of the proposed estimators with respect to various criteria quantities using Monte Carlo simulations. Further, using different optimality criteria, an optimal censoring scheme has been suggested. Besides, one-sample and two-sample prediction problems based on observed sample and predictive intervals under Bayesian framework are discussed. Finally, to demonstrate the proposed methodology in a real-life scenario, maximum flood level data is considered to show the applicability of the proposed methods.

Inference for Generalized Inverted Exponential Distribution UnderProgressive Type-I Interval Censored Data

This article discusses the estimation methods for parameters of a generalized inverted exponential distribution with different estimation methods by using Progressive type-I interval censored data. In addition to conventional maximum likelihood estimation, the mid-point method, probability plot method and method of moments are suggested for parameter estimation. To get maximum likelihood estimates, we utilize the Newton-Raphson, expectation -maximization and stochastic expectation-maximization methods. Furthermore, the approximate confidence intervals for the parameters are obtained via the inverse of the observed information matrix. The Monte Carlo simulations are used to introduce numerical comparisons of the proposed estimators. In addition, we use the percentile bootstrapping technique that is used to calculate confidence intervals. The proposed methodology in a real-life using the survival times of guinea pigs inoculated with different doses of tubercle bacilli data are considered to offer the applicability of the suggested methods.  

Box-Cox Transformation for Exponential Smoothing With Application

This article introduces a novel algorithm for incorporating power transformation into the estimation process of a Holt-Winters Seasonal model. The algorithm outlines a series of steps aimed at selecting the most appropriate power parameter estimate. This selection is achieved using the conventional Maximum Likelihood Estimation method in combination with various criteria for enhancing statistical modeling efficiency. Supplementary decision rules include assessing Mean Square Error, Mean Absolute Error, and conducting a p-value test for the normality of errors. The algorithm's effectiveness is demonstrated through its application to real-world data. Ultimately, the article affirms the feasibility of obtaining viable solutions for selecting the optimal power parameter.

Develop a Nonlinear Regression Model Using Box-Cox Transformation with Application

This article introduces an algorithm designed for utilizing power transformations in the estimation of nonlinear regression models. The algorithm outlines a series of steps for selecting the most suitable power parameter estimate through a combination of the conventional Maximum Likelihood Estimation technique and specific criteria for enhancing statistical modeling effectiveness. Supplementary decision guidelines involve the utilization of the determination coefficient and the p-value from the errors normality test. The algorithm's application was demonstrated using actual data. The article's conclusion highlighted the ability to identify a range of feasible solutions for selecting the optimal power parameter. However, it was acknowledging the challenge of identifying a single optimal value that satisfies the requirements of all estimation and decision methodologies.

Neural Network-Based Wind Measurements in Rainy Conditions Using the HY-2A Scatterometer

Wind measurement using spaceborne scatterometers has been used for various scientific and operational purposes. However, the major problem of such measurements is contamination by rain. To improve the wind measurement using the HY-2A scatterometer under rainy conditions, a neural network-based model was established in this study. The model is almost autonomous in that it only needs the backscatter coefficient measurement data and the observation geometry information from the HY-2A scatterometer itself. The model can distinguish between rain-contaminated wind pixels and rain-free wind pixels and significantly improve the accuracy of wind speed measurements using HY-2A scatterometer alone. TAO data and linearly calibrated ECMWF data were used in the study to validate the neural network-inverted wind speed. Under no rain conditions, the RMS of the neural network-inverted wind speed and TAO wind speed was 1.06 m/s, with a deviation of −0.21 m/s, which is a small difference from the standard method inverted wind speed. Under rain conditions, the RMS and deviation were 1.94 m/s and 0.66 m/s, respectively, which were better than the statistical results of the conventional maximum likelihood estimation method. The validated results using linearly calibrated data also indicate that the neural network-inverted wind speed is closer to the validation data under rain conditions.

Constraint Linear Model for Period Estimation and Sparse Feature Extraction Based on Iterative Likelihood Ratio Test

This study proposes a constraint linear model (CLM) to represent periodic signals. Different from the conventional linear model, linear constraints are added to the proposed model to fulfill the continuity constraints of signals. To estimate the period of the signals, a hypothesis testing based on the generalized likelihood ratio is proposed to statistically compare misleading peaks exhibited in the likelihood function of CLM. The misleading peaks usually arise because the likelihood function has a similar value in the true period and its multiples, and this misleading effect obstructs the effectiveness of the conventional maximum likelihood estimation method. An iterative likelihood ratio test (ILRT) is proposed, in which hypothesis testing is iteratively conducted to test the significance of an identified period until no candidate period with significant evidence is found. After the period is estimated by ILRT, an L1 penalty is added to CLM to construct a sparse representation of the signals, and an augmented Lagrangian shrinkage algorithm is applied to extract sparse features from the signals. The effectiveness of the proposed method is verified through a simulation study on synthetic signals and a case study on real vibration signals.

Performance of the Ridge and Liu Estimators in the zero-inflated Bell Regression Model

The Poisson regression model is popularly used to model count data. However, the model suffers drawbacks when there is overdispersion—when the mean of the Poisson distribution is not the same as the variance. In this situation, the Bell regression model fits well to the data. Also, there is a high tendency of excess zeros in the count data. In this case, the zero-inflated Bell regression model is an alternative to the Bell regression model. The parameters of the zero-inflated Bell regression model are mostly estimated using the method of maximum likelihood. Linear dependency is a threat in a real-life application when modeling the relationship between the response variable and two or more explanatory variables in a generalized linear model such as the zero-inflated Bell regression model. It reduced the efficiency of the maximum likelihood estimator. Therefore, we developed the ridge and Liu estimators for the zero-inflated Bell regression model to deal with this issue. The simulation and application results support the dominance of the proposed methods over the conventional maximum likelihood estimator.

An Improved Parameter Estimator of the Homodyned K Distribution Based on the Maximum Likelihood Method for Ultrasound Tissue Characterization.

The homodyned K distribution (HK) can generally describe the ultrasound backscatter envelope statistics distribution with parameters that have specific physical meaning. However, creating robust and reliable HK parameter estimates remains a crucial concern. The maximum likelihood estimator (MLE) usually yields a small variance and bias in parameter estimation. Thus, two recent studies have attempted to use MLE for parameter estimation of HK distribution. However, some of the statements in these studies are not fully justified and they may hinder the application of parameter estimation of HK distribution based on MLE. In this study, we propose a new parameter estimator for the HK distribution based on the MLE (i.e., MLE1), which overcomes the disadvantages of conventional MLE of HK distribution. The MLE1 was compared with other estimators, such as XU estimator (an estimation method based on the first moment of the intensity and tow log-moments) and ANN estimator (an estimation method based on artificial neural networks). We showed that the estimations of parameters α and k are the best overall (in terms of the relative bias, normalized standard deviation, and relative root mean squared errors) using the proposed MLE1 compared with the others based on the simulated data when the sample size was N = 1000. Moreover, we assessed the usefulness of the proposed MLE1 when the number of scatterers per resolution cell was high (i.e., α up to 80) and when the sample size was small (i.e., N = 100), and we found a satisfactory result. Tests on simulated ultrasound images based on Field II were performed and the results confirmed that the proposed MLE1 is feasible and reliable for the parameter estimation from the ultrasonic envelope signal. Therefore, the proposed MLE1 can accurately estimate the HK parameters with lower uncertainty, which presents a potential practical value for further ultrasonic applications.

Federated Edge Learning With Misaligned Over-the-Air Computation

Over-the-air computation (OAC) is a promising technique to realize fast model aggregation in the uplink of federated edge learning (FEEL). OAC, however, hinges on accurate channel-gain precoding and strict synchronization among edge devices, which are challenging in practice. As such, how to design the maximum likelihood (ML) estimator in the presence of residual channel-gain mismatch and asynchronies is an open problem. To fill this gap, this paper formulates the problem of misaligned OAC for FEEL and puts forth a whitened matched filtering and sampling scheme to obtain oversampled, but independent samples from the misaligned and overlapped signals. Given the whitened samples, a sum-product ML (SP-ML) estimator and an aligned-sample estimator are devised to estimate the arithmetic sum of the transmitted symbols. In particular, the computational complexity of our SP-ML estimator is linear in the packet length, and hence is significantly lower than the conventional ML estimator. Extensive simulations on the test accuracy versus the average received energy per symbol to noise power spectral density ratio (EsN0) yield two main results: 1) In the low EsN0 regime, the aligned-sample estimator can achieve superior test accuracy provided that the phase misalignment is not severe. In contrast, the ML estimator does not work well due to the error propagation and noise enhancement in the estimation process. 2) In the high EsN0 regime, the ML estimator attains the optimal learning performance regardless of the severity of phase misalignment. On the other hand, the aligned-sample estimator suffers from a test-accuracy loss caused by phase misalignment.

Dawoud–Kibria Estimator for Beta Regression Model: Simulation and Application

The linear regression model becomes unsuitable when the response variable is expressed as percentages, proportions, and rates. The beta regression (BR) model is more appropriate for the variable of this form. The BR model uses the conventional maximum likelihood estimator (BML), and this estimator may not be efficient when the regressors are linearly dependent. The beta ridge estimator was suggested as an alternative to BML in the literature. In this study, we developed the Dawoud–Kibria estimator to handle multicollinearity in the BR model. The properties of the new estimator are derived. We compared the performance of the estimator with the existing estimators theoretically using the mean squared error criterion. A Monte Carlo simulation and a real-life application were carried out to show the benefits of the proposed estimator. The theoretical comparison, simulation, and real-life application results revealed the superiority of the proposed estimator.

Empirical Bayes estimation of pairwise maximum entropy model for nonlinear brain state dynamics

The pairwise maximum entropy model (pMEM) has recently gained widespread attention to exploring the nonlinear characteristics of brain state dynamics observed in resting-state functional magnetic resonance imaging (rsfMRI). Despite its unique advantageous features, the practical application of pMEM for individuals is limited as it requires a much larger sample than conventional rsfMRI scans. Thus, this study proposes an empirical Bayes estimation of individual pMEM using the variational expectation-maximization algorithm (VEM-MEM). The performance of the VEM-MEM is evaluated for several simulation setups with various sample sizes and network sizes. Unlike conventional maximum likelihood estimation procedures, the VEM-MEM can reliably estimate the individual model parameters, even with small samples, by effectively incorporating the group information as the prior. As a test case, the individual rsfMRI of children with attention deficit hyperactivity disorder (ADHD) is analyzed compared to that of typically developed children using the default mode network, executive control network, and salient network, obtained from the Healthy Brain Network database. We found that the nonlinear dynamic properties uniquely established on the pMEM differ for each group. Furthermore, pMEM parameters are more sensitive to group differences and are better associated with the behavior scores of ADHD compared to the Pearson correlation-based functional connectivity. The simulation and experimental results suggest that the proposed method can reliably estimate the individual pMEM and characterize the dynamic properties of individuals by utilizing empirical information of the group brain state dynamics.

Predicting Individual Effects in Fixed Effects Panel Probit Models

Abstract Many applied settings in empirical economics require estimation of a large number of individual effects, like teacher effects or location effects; in health economics, prominent examples include patient effects, doctor effects or hospital effects. Increasingly, these effects are the object of interest of the estimation, and predicted effects are often used for further descriptive and regression analyses. To avoid imposing distributional assumptions on these effects, they are typically estimated via fixed effects methods. In short panels, the conventional maximum likelihood estimator for fixed effects binary response models provides poor estimates of these individual effects since the finite sample bias is typically substantial. We present a bias-reduced fixed effects estimator that provides better estimates of the individual effects in these models by removing the first-order asymptotic bias. An additional, practical advantage of the estimator is that it provides finite predictions for all individual effects in the sample, including those for which the corresponding dependent variable has identical outcomes in all time periods over time (either all zeros or ones); for these, the maximum likelihood prediction is infinite. We illustrate the approach in simulation experiments and in an application to health care utilization.

Bayesian estimation of maximum entropy model for individualized energy landscape analysis of brain state dynamics.

The pairwise maximum entropy model (MEM) for resting state functional MRI (rsfMRI) has been used to generate energy landscape of brain states and to explore nonlinear brain state dynamics. Researches using MEM, however, has mostly been restricted to fixed‐effect group‐level analyses, using concatenated time series across individuals, due to the need for large samples in the parameter estimation of MEM. To mitigate the small sample problem in analyzing energy landscapes for individuals, we propose a Bayesian estimation of individual MEM using variational Bayes approximation (BMEM). We evaluated the performances of BMEM with respect to sample sizes and prior information using simulation. BMEM showed advantages over conventional maximum likelihood estimation in reliably estimating model parameters for individuals with small sample data, particularly utilizing the empirical priors derived from group data. We then analyzed individual rsfMRI of the Human Connectome Project to show the usefulness of MEM in differentiating individuals and in exploring neural correlates for human behavior. MEM and its energy landscape properties showed high subject specificity comparable to that of functional connectivity. Canonical correlation analysis identified canonical variables for MEM highly associated with cognitive scores. Inter‐individual variations of cognitive scores were also reflected in energy landscape properties such as energies, occupation times, and basin sizes at local minima. We conclude that BMEM provides an efficient method to characterize dynamic properties of individuals using energy landscape analysis of individual brain states.

Frequency estimation of a single real-valued sinusoid: An invariant function approach

An invariant function approach for the computationally efficient (non-iterative and gridless) maximum likelihood (ML) estimation of unknown parameters is applied on the real-valued sinusoid frequency estimation problem. The main attraction point of the approach is its potential to yield a ML-like performance at a significantly reduced computational load with respect to conventional ML estimator that requires repeated evaluation of an objective function or numerical search routines. The numerical results indicate that the suggested estimator closely tracks the Cramer-Rao bound in the asymptotic region and performs very close to the ML estimator in other regions.

Parameter optimization and uncertainty assessment for rainfall frequency modeling using an adaptive Metropolis-Hastings algorithm.

A new parameter optimization and uncertainty assessment procedure using the Bayesian inference with an adaptive Metropolis-Hastings (AM-H) algorithm is presented for extreme rainfall frequency modeling. An efficient Markov chain Monte Carlo sampler is adopted to explore the posterior distribution of parameters and calculate their uncertainty intervals associated with the magnitude of estimated rainfall depth quantiles. Also, the efficiency of AM-H and conventional maximum likelihood estimation (MLE) in parameter estimation and uncertainty quantification are compared. And the procedure was implemented and discussed for the case of Chaohu city, China. Results of our work reveal that: (i) the adaptive Bayesian method, especially for return level associated to large return period, shows better estimated effect when compared with MLE; it should be noted that the implementation of MLE often produces overy optimistic results in the case of Chaohu city; (ii) AM-H algorithm is more reliable than MLE in terms of uncertainty quantification, and yields relatively narrow credible intervals for the quantile estimates to be instrumental in risk assessment of urban storm drainage planning.

Robust Demand Estimation of Choice Models for Personalized Offers Using Sales Transaction Data

As firms come to realize that a traditional one-size-fits-all policy may no longer be effective, they look for more innovative practices such as personalized offerings and even personalized pricing to differentiate their products and services. In such an environment, it becomes critical to understand customer preferences and estimate customer choice among a firm's portfolio of offerings when the prices of those offerings vary over time and sometimes even across different customers. We develop a novel statistical method to estimate the choice probabilities and the size of the no-purchase customer population when transaction data from a single firm's set of products is available. We propose a conditional logit model to fit this data that does not assume a constant arrival rate and allows for choice sets and product attributes that can vary across each customer arrival, unlike existing methods which require some level of aggregation across arrivals and/or choice sets. Customers independently arrive to the system through a non-stationary process to choose a product among several options or choose not to buy any product. Although the parameters of our proposed model can be consistently estimated using conventional maximum likelihood estimation, the no-purchase utility cannot be estimated without further information. We consider two additional types of information for identification of our model parameters: 1) additional assumptions on the customers' utility function, and 2) external information about a firm's market share. We then develop a robust estimation procedure that accounts for inaccuracies in either information type and lets the data determine the best approach. Computational experiments show that our approach provides promising predictions of customer choice behavior when compared with other generally used methods, and clearly outperforms those methods in scenarios where the product prices change frequently over time. Relative to existing approaches for estimating customer choice-based models, our proposed methodology better suits environments employing dynamic pricing and personalized offering practices, such as online retailing.

Bayesian non-linear regression with spatial priors for noise reduction and error estimation in quantitative MRI with an application in T1 estimation

Purpose. To develop a method that can reduce and estimate uncertainty in quantitative MR parameter maps without the need for hand-tuning of any hyperparameters. Methods. We present an estimation method where uncertainties are reduced by incorporating information on spatial correlations between neighbouring voxels. The method is based on a Bayesian hierarchical non-linear regression model, where the parameters of interest are sampled, using Markov chain Monte Carlo (MCMC), from a high-dimensional posterior distribution with a spatial prior. The degree to which the prior affects the model is determined by an automatic hyperparameter search using an information criterion and is, therefore, free from manual user-dependent tuning. The samples obtained further provide a convenient means to obtain uncertainties in both voxels and regions. The developed method was evaluated on T 1 estimations based on the variable flip angle method. Results. The proposed method delivers noise-reduced T 1 parameter maps with associated error estimates by combining MCMC sampling, the widely applicable information criterion, and total variation-based denoising. The proposed method results in an overall decrease in estimation error when compared to conventional voxel-wise maximum likelihood estimation. However, this comes with an increased bias in some regions, predominately at tissue interfaces, as well as an increase in computational time. Conclusions. This study provides a method that generates more precise estimates compared to the conventional method, without incorporating user subjectivity, and with the added benefit of uncertainty estimation.

ACADEMIC CHANGE-ORIENTED LEADERSHIP CAPABILITY: THE APPLICATION OF SATORRA-BENTLER ROBUST METHOD IN A SCALE VALIDATION STUDY

This research aims at validating a previously developed 5-factor model of change-oriented leadership capability scale in academic settings. Data were collected from Malaysian academic leaders in 25 public and private universities through an online survey distribution platform. The result of data screening procedure highlighted the existence of one outlying case, which was removed from the dataset. In addition, the assumption of multivariate normality was not met. Hence, to run a confirmatory analysis to test the factorial validity of the proposed scale, we applied the robust Maximum-Likelihood (ML) methodology in EQS 6.4 software package to correct the chi-square statistic and generate robust standard errors. The outcome of the analysis shed light on the fact that the change-oriented capability scale developed in academic settings was a 4-factor model. In addition, the fit indices and standard errors generated through conventional ML estimator and the robust method were compared to provide more insights regarding the undue influence of the multivariate non-normal data on fit indices and standard errors. The implications of the findings with respect to the practice, theory, and methodology were discussed as well.

Detection and analysis of Cd in soil by laser-induced breakdown spectroscopy based on maximum likelihood estimation

Cd is an extremely toxic element and has a great impact on human health. But the detection of Cd in the soil is still a challenge. This study aimed to improve the analytical ability of laser-induced breakdown spectroscopy (LIBS) to measure Cadmium, the trace elements in the soil. Twelve soil samples containing gradient Cd elements were prepared. The sample concentration was quantitatively analyzed and compared by using the conventional univariate calibration model and maximum likelihood estimation algorithm. Calibration analysis revealed that the fitted R2 values are 0.965 and 0.990 by the conventional univariate calibration and maximum likelihood (ML) estimation respectively. The ML estimation has higher stability and predicted concentration is more accurate. Finally, the detection limit of Cd was estimated to be 7.84 μg/g by using the proposed algorithm to correct the intensity.